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Method and apparatus for the determination of the contour 
of sheet metal blanks 



Field of the Invention 

This invention relates to a photogrammetry method and appara- 
tus. In particular, the invention relates to a method and ap- 
5 paratus for the determination of the contour of sheet metal 
blanks . 

Background of the Invention 

10 For an average passenger car about 250 different blanks are 
needed. The blanks have to be cut of sheet metal coils.. De- 
pending on the shape complexity, this can be done by swivel- 
mounted shears for angular straight cuts, adjustable dies 
with variable pitch for parallel cuts, or by form dies for 

15 blanks with a complicated contour. By use of form dies and 
appropriate nesting on the coil material usage can be mini- 
mized. However, tooling costs increase and are not justified 
for simple shapes. Typically 50-70 of the total 250 parts 
will be cut by form dies - 

20 

In principle, the blank follows the binder contour of the 
press tool as it was designed by die systems engineering. 
However, material flow rate is not equally distributed along 
the binder due to dynamic effects, material properties and 
25 process parameters. Lock beads may be inserted to avoid folds 
and to influence material hardness. Therefore, the binder ge- 
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ometry is insufficient to determine the shape of the form 
dies for blank cutting. Although soft tooling experiments and 
FEM computations can help in predicting the blank contour, 
only tool tryout will deliver conclusive results. 

5 

During the blank engineering process for a new car line, the 
manually optimized shape has to be digitized for tool con- 
struction and blank nesting. Since the sheet metal is two- 
dimensional the digitization task is not complicated in terms 
10 of technology used. The critical factor is time needed to 
digitize the 50 - 70 parts that require form dies. In the 
state of the art, the following techniques are used for this 
task : 



15 - coordinate measuring machine (CMM) , 

paper copy and digitization device, and 

paper copy and manual measurement (re-engineering) . 

These techniques are inherently slow. The accuracy of CMMs is 
20 higher than adequate for this task. Since they are not pro- 
vided for such relatively low precision work they will often 
not be available. Even so, the 3D point data of a CMM has to 
be converted into a preferred CAD format. The start point of 
the two other methods is a paper copy of the blank. The out- 
25 line of the blank is simply transferred by a pencil on a 
large sheet of paper. In a second step a drawing table with a 
built-in electronic digitization device can be used. Points 
on the contour can be picked and digitized by pressing a 
trigger button. Again, this data has to be converted into the 
30 preferred CAD drawing format. As an alternative, the paper 
sketch of the blank can be used for part re-engineering. Part 
dimensions can be measured by hand on paper and are then 
keyed in a 2D drawing program to reconstruct the blank. 
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Equipment such as drawing tables has disappeared with the in- 
troduction of CAD and closed-loop CAD /CAM environments and is 
only available through external services. The access to ex- 
5 ternal suppliers will additionally slow down the process. 

Taking the mentioned drawbacks of current state of the art 
into account, the objective of the present invention was to 
achieve cost reductions and an improved timing in the produc- 
10 tion of sheet metals for cars. 

Summary of the Invention 

This objective is achieved by methods according to claims 1 



30 



15 and 8 and an apparatus according to claim 10. Preferred em- 
bodiments are subject of the dependent claims. 

The method according to the present invention allows the de- 
termination of the contour of substantially planar workpieces 
20 and comprises the following steps: 



a) preparing the workpiece and possibly the workpiece sup- 
port by applying a plurality of length scales and/or po- 
sition-markers that are spread out over the surface of 

25 the workpiece and/or its support; 

b) taking at least two overlapping photographs of the pre- 
pared workpiece from various perspectives with a digital 
camera; 



c) photogrammetrical processing of the photographs with a 
data processing unit for producing a true- to-scale over- 
all image of the workpiece, and 
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d) determining the contour of the workpiece from the true- 
to-scale overall image produced in step c) . 

5 The cited method relies on the use of a digital camera for 
data acquisition and the use of photogrammetry for the proc- 
essing of these data. For the preferred application of this 
method, in which the workpieces are metal blanks for cars, 
considerable savings in time and material can be achieved by 

10 

time savings for digitization of hand optimized blank 
contours, and 

- improved blank nesting. 

15 

The underground of the workpiece during the taking of the 
photos is preferably dark (or even black) in comparison to 
the workpiece. This guarantees a high contrast between back- 
ground and workpiece and helps to determine the contour. 

20 

In order to increase contrast between background and work- 
piece and to avoid impairing reflections, the workpiece may 
be coated with a contrasting layer before taking the photos. 
The workpiece may e.g. be sprayed with a matt paint. 

25 

The position-ma rkers and^rulers that are arranged on the sup- 
port of the workpiece during the photography are preferably 
disposed on equalizing or compensation layers having a thick- 
ness corresponding to the thickness of the workpiece. Thus, 
30 they are in the same vertical height as the markers and rul- 
ers on the workpiece, which allows a more precise determina- 
tion and reproduction of their position and measures in the 
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photos. Moreover, weights may be positioned on the workpiece 
in order to press it flat on the ground. 

According to a preferred embodiment of the invention, at 
least two photos of a workpiece are taken from perspectives 
that are substantially orthogonal to each other. Such photo- 
graphs allow for a most precise reproduction of measures and 
positions of the workpiece during photogrammetry . 

The photos of the workpiece may be rectified. This means that 
the image plane and the workpiece plane are transformed onto 
each other. After rectification, the different single photos 
can readily be combined to form a complete picture of the 
whole workpiece . 

The contour that is determined as a result of the proposed 
method is preferably defined as a polygon. Such a polygon re- 
quires a minimal amount of data and at the same time allows 
the approximation of a real shape with any desired precision. 

Moreover, the invention concerns a method of establishing a 
form die for cutting out sheet metal parts, wherein in a plu- 
rality of respective steps : 

a prototype of the form die is produced, 

a test sheet is cut out with the prototype, 

the contour of the test sheet is determined and compared 
to a reference contour, and 

the shape of the next prototype of the form die is ad- 
justed on the basis of the comparison. 
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The method is characterized in that the contour of the test 
sheet is determined with a method described above. 

5 Due to the easy and rapid acquisition of the contour of the 
blanks during this method, the considerable savings of time 
and material mentioned above can be achieved. 

The invention also comprises an apparatus for the determina- 
10 tion of the contour of workpieces in a method explained 
above. With such an apparatus, the advantages of the method 
can be realized. The apparatus comprises: 

position-markers and length scales for application to the 
15 workpiece and/or the workpiece support; 

a digital camera for recording digital, electronically 
stored photographs of the workpiece; and 

20 - a data processing unit which is adapted for photogrammet- 
rical processing of photographs of a workpiece recorded 
with the digital camera, for producing a true- to-scale 
overall image of the workpiece therefrom and for deter- 
mining the contour of the workpiece from the true-to- 

25 scale overall image. 

Brief description of the drawings 

In the following the invention will be described exemplary 
30 with reference to the accompanying drawings in which: 



Fig. 1 shows the scheme of convergent photogrammetry ; 
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illustrates the situation of pinhole projection; 

shows rulers, background material and coded targets 
used to prepare a workpiece for photography; 

illustrates the workflow from single photos of a 
workpiece to a CAD model; 

illustrates the repeatability of measurements 
according to the present invention; 

illustrates a situation that reveals the limits of 
conventional CMM contour tracking; 

illustrates the absolute accuracy of the inventive 
method. 

Detailed Description of the Preferred Embodiment 

20 In the production of sheet metal blanks for cars, a certain 
percentage of the blanks used in the press shops are not 
bounded by straight cuts. These so-called contour blanks are 
cut by form dies to optimize material usage of sheet metal 
coils- During the tryout process of the press tools the final 

25 contour for the blanks is determined in an iterative process. 
The manually size-optimized blank contour has to be digitized 
for blank nesting and the construction of a form die to cut 
the blank from the coil in production. According to the pre- 
sent invention, a low-cost measurement system based on a 

30 digital camera is proposed. It allows to take pictures of 
blanks on-site in the plant and to process them off-line to 
regenerate geometrical information by use of photogrammetric 
principles. This procedure provides a significant reduction 



10 



Fig. 2 
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Fig. 4 

Fig. 5 

Fig. 6 



15 Fig. 7 
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of lead-time by eliminating the need for a slow coordinate 
measuring machine (CMM) . 

The primary use of photogrammetry is in the development of 
5 topographic maps using aerial photographs. Depending on the 
accuracy requirements and the application environment, one 
can distinguish three types of photogrammetric devices: 



Metric cameras: 



10 



They have stable and precisely known internal geometry 
and very low lens distortions. Therefore, they are very 
expensive devices. Today, all of them have an image for- 
mat of 23 by 23 centimeters and they are often used in 
aerial photogrammetry . 



15 



Stereometric cameras : 



20 



A stereometric camera in principle consists of two metric 
cameras mounted at both ends of a bar. Both photographs 
have viewing directions which are parallel to each other 
and in a right angle to the mounting base, simulating hu- 
man stereoscopic vision. 



"Amateur 



ii 



cameras : 



25 



An "amateur camera" is a photographic device of which the 
internal geometry is unstable or unknown, as is the case 
with any "normal" commercially available camera. However; 
also these can be very expensive and technically highly 
developed. Photographing a test field with many control 
points, a "calibration" of the camera can be calculated. 



30 



However, the precision will not reach that of metric cam- 



eras . 
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A photographic image is a "central perspective". This implies 
that every light ray which reached the film surface passed 
through the camera lens (which is mathematically considered 
as a single point, the "perspective center"). In order to 
5 take measurements of object points from photographs, the ray 
bundle must be reconstructed. Therefore, the internal geome- 
try of the used camera (which is defined by the focal length, 
the position of the principal point and the lens distortion) 
has to be precisely known. The principal point is the coordi- 
10 nate where the axis through the perspective center hits the 
film or CCD plane. The focal length is called "principal dis- 

p tance", which is the distance of the projection center from 

jK the image plane's principal point. 

?5 15 Figure 1 shows the taking of photographs of a metal blank 2 0 
2 with a digital camera 21. By taking pictures from at least 

= two different locations and measuring the points of interest 

S (image points 22) in each photograph, one can develop lines 

rU of sight 23 from each camera location to the points of inter - 

5 20 est 24 on the object. The intersection of these pairs of 
^" lines of sight can then be triangulated to produce the 

three-dimensional coordinate of the point 24 on the object. 

In this way, a pair of two-dimensional measurements of the u, 

v positions of the point in each photograph is used to pro- 
25 duce the single x, y, z coordinate measurement of the point 

on the object. In Figure 1, the optical axis of the camera is 

referenced with number 25. 

Unlike stereo photogrammetry , the convergent photogrammetry 
30 process does not attempt to imitate the stereoscopic viewing 
capabilities of the human vision system to make measurements. 
Instead, photographs are taken with the camera axes typically 
inclined towards each other (rather than parallel to each 
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other as with the stereo method) so that the camera axes con- 
verge or intersect as shown in Figure 1. In other words, the 
viewing directions of the camera must not be parallel for all 
pictures. One now measures easily distinct features in each 
5 photograph, e.g. corner points, and these measurements are 
combined together to produce the three-dimensional coordi- 
nates of the points. The convergent method is not limited to 
using just two photographs of an object at a time. Many pho- 
tographs can be taken which leads to higher accuracy and re- 
10 liability and makes it far easier to measure complex objects 
which cannot be completely seen in just two photographs. Us- 
ing three and more photographs allows to compute coordinates 
with a minimum square error adjustment and to estimate errors 
for each point, providing a quality indication of the proc- 



The mathematics behind this process can best be explained by 
the simple pinhole camera model illustrated in Figure 2 . The 
pinhole camera model connects the 3D world point (x,y,z) with 

20 the 2D image point (u,v) by a straight line, neglecting all 
lens effects. A perspective projection is the projection of a 
three-dimensional object 30 onto a two-dimensional surface 31 
(consisting of pixels 33) by straight lines that pass through 
a single point, the center of projection 32. Geometric rela- 

25 tions show that given the distance f of the image plane 31 to 
the center of projection 32, then the image coordinates 
(Ui,yi) are related to the object coordinates (xo,y 0 /Zo) by 



15 



ess . 



Ui = U + U c = (f/z 0 )'X 0 + U c 



30 



and 



Vi = v + v c = (f/z 0 )-y 0 + v c 
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or the object distance z expressed as scale factor s and 
written as a homogeneous matrix equation: 



su 
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5 

where (u C/ v c ) is the image center point and the optical axis 
z is assumed to pass through the image center (Figure 2) . The 
transf ormation of (+u c/ +v c ) accounts for the fact that image 
coordinates in pixels 33 are by convention given as positive 

10 integer numbers. The pixel coordinates can be obtained from 
the above equation by dividing u and v by the physical pixel 
size of the applied image sensor. Current digital technology 
often uses 2/3' ' sensors (8.576 x 6.861 mm 2 ) with a pixel 
element size of 6.7 ^im 2 . In a real application the z-axis 

15 will never pass exactly through the center of the image since 
it is impossible to adjust lenses and the sensor with a pre- 
cision of only several microns. Therefore, the true point of 
intersection between image sensor and optical axis, the so- 
called principal point, has to be determined by the camera 

20 calibration process. 

The parameters pixel size, focal length and principal point 
are called intrinsic camera parameters since they describe 
the internal camera geometry and are not dependent on the po- 
25 sition and orientation of the camera in space. As an exten- 
sion to the simplified pinhole model lens distortions have to 
be considered in a real application as a major influence on 
the projection. Very often the camera model presented by Tsai 
is used (Tsai, R. , Y. : A versatile camera calibration tech- 
* 30 nique for high accuracy 3D machine vision metrology using off 
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the self TV cameras and lenses; in: IEEE Journal of Robotics 
and Automation, Vol.3, No. 4, 1987). 




In general, the three-dimensional world coordinates of a 
5 point will not be specified in a frame whose origin is at the 
center of projection and whose z axis lies along the optical 
axis. Some more convenient coordinate frame will more likely 
be chosen and a general transformation matrix has to be in- 
cluded in the projection equations to describe the camera ro- 
10 tation and orientation with respect to that frame. 



O In order to achieve a high degree of automation, reliability 

ff. and accuracy in the measuring process, one normally measures 

yj high-contrast targets placed on or near the points of inter - 

S3 

Qi 15 est on the object. Since the geometry of the targets is known 

Jrf (e.g. a circle), they can be detected in the image with so 

- called subpixel accuracy. Interpolation techniques are used 
to find the center point position of the target in the images 

i_U with higher resolution than offered by the camera. Therefore, 

rii » 

p 20 target points can be detected with highest accuracy. Accuracy 
^ specifications of vendors are mostly related to the detection 

of target points. The coordinates of natural points of the 
image like edges, corners or holes can be computed with lower 
accuracy as targets (marked or signaled points) . Correspond- 
25 ing natural points in different images have to be specified 
manually (e.g. by a mouse click). This process cannot be done 
with subpixel accuracy. 



In the frame of the present invention, coordinates of natural 
30 points, namely edges of the hand optimized blank, have to be 
digitized- Marked points or targets are only used to register 
partially overlapping images relative to each other and to 
work out the transform from image to world coordinates. 
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One of the objectives for a blank photogrammetry system is 
ease of use with no delicate equipment or tedious processes. 
A major drawback of traditional photogrammetry was the com- 
plicated and slow film development with an additional film 
scanning procedure to analyze images by software on a com- 
puter. This process has been dramatically cut by the avail- 
ability of digital cameras. Here, a CCD (charge coupled de- 
vice) replaces the film in a standard camera housing. The 
electronic device captures the image in a digital form so 
that they can directly be transferred to a computer. 

There are a couple of technical issues to consider before se- 
lecting a consumer ("amateur") digital camera for measurement 
purposes. An automatic fine adjustment of the focus (autofo- 
cus) will lead to erroneous camera calibration parameters and 
decrease measurement accuracy. A camera chip with square pix- 
els is advantageous because it avoids differences in scaling 
for x and y image direction. Fixed focal length lenses of 
high quality are preferable and a stable mechanical housing 
is required. 

Before use in a photogrammetry application the camera has to 
be calibrated. In particular, the focal length in a certain 
position of the zoom lens has to be known. Most photogram- 
metry software can include the camera parameters in the 
mathematical optimization which decreases the impact of mis- 
calibration. This process is called simultaneous calibration. 
However, a good starting point for the camera parameters is 
essential to get reliable results from simultaneous calibra- 
tion. In this study, a free software from the university of 
Oulu, Finland was used for calibration (cf. Heikkila, J., 
Silven, 0.: Calibration procedure for short focal length off- 
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the-shelf CCD cameras; in: Proc of The 13th International 
Conference on Pattern Recognition. Vienna, Austria 1996, pp. 
166-170). It is a series of scripts for the MatLab computing 
environment and available over the internet. 

5 

Experiments with a motorized Zoom lens Consumer Camera showed 
that the principle point is varying when the camera is closed 
(zoom lens retracted). The reason for this is the backlash of 
the motor driven mechanics. This clearly indicates that a 

10 zoom camera is not the optimal device for measuring applica- 
tions. In production, a fixed focus objective should be used. 
However, for the test it was possible to calibrate the cam- 
era, let it open until the final shots had been made, and in- 
clude the camera parameters in the photogrammetric optimiza- 

15 tion. 

In summary, new digital cameras offer the opportunity to per- 
form geometric measurements by photogrammetry with low effort 
and at low cost compared to film cameras or metric cameras. 

20 

The main criteria for the selection of photogrammetric soft- 
ware are : 

ease-of-use 

25 

suitability for the task, work speed 

price, including software adaptations if necessary. 



30 The first step in any photogrammetric project is to collect 
data in the field. This involves two main tasks: providing a 
network of target points to establish a reference system be- 
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tween the photographs and the real world, and taking the ac- 
tual photographs of the project area. 

Targets are needed as reference points to calculate the loca- 
tion and orientation of a set of photographs. Since the tar- 
gets can be detected with high accuracy, they should also be 
used to provide distance information. The coded targets are 
: arbitrarily placed on the blank and its surroundings. ^The 
j ^whole i mage^ area should ^contain targets . As distance refer- 
10 T ence objects rulers 4*3"* "("Fig". 3) were used with targets pre- 
l cisely attached to it. 



The best background for the application is black paper 42 or 
fabric. Depending on the surface and flash conditions the 

15 sheet metal can appear as very bright (mirror-like) or dark 
(rust or in image corner) . There can be problems to distin- 
guish sheet metal and background if the background color is 
not black. Shadowing effects at the edge of the blank have no 
effect on the black background. All items necessary to pre- 

20 pare the workpieces are shown in Fig. 3: black background ma- 
terial 42, coded targets 41 and coded rulers 43 of known 
length. 

For the first three images of a project a standard direction 
25 to the workpiece coordinate system is recommended (view left, 
view right, front view) - The views from left and right should 
intersect at a large angle of about 90°. This procedure sta- 
bilizes the resulting equation system. The rest of the images 
can be taken as feasible. If the workpiece is placed on the 
30 floor one should walk around it and take pictures from all 
sides. Each photograph must contain a fair a mount of overlap 



with some of the other images. This overlap will be used in 



the analysis phase to calculate relative orientation between 
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images- The scale, orientation, location, etc. of the images 
to each other is not critical, nor is it recorded in any way 
in the field. For large objects, it is advisable to end the 
photo session with some close-ups from the top that provide a 
5 good view on a part of the blank. Two or three of these final 
shots will be the photos that are actually used to trace the 
contour. The others, in particular all long range overview 
image, only stabilize the coordinate information. 

10 The process of reading the project image files, identifying 
corresponding points, computing relative and absolute orien- 
m tation and bundle adjustment can be fully automated. After 

M«l the image orientation and camera positions have been found, 

rr= 

=~==r 

tg world coordinates can be measured. Rolleimetric CDW allows 

2; 15 working in a plane, i.e. the coordinates are computed by in- 
Ep tersection of the line of sight with the workpiece plane. 

_ There is no specification of identical points in multiple im- 

2f ages necessary. The workpiece contour can be simply traced by 

Fy a drawing tool. A part of the image can be magnified to work 

20 as accurate as possible. These tools are well conceived and 
M 8 work efficiently with a few mouse clicks. The generated poly- 

gon can be exported as DXF or ASCII point list. 

Very often the complete workpiece contour will be patched by 
25 several pictures to increase resolution. The contour would 
have to be traced in several pictures and the corresponding 
partial polygons are not closed connected. The gaps between 
starting points and end points would have to be connected in 
a CAD system before the result can be processed by a nesting 
30 algorithm. The rectification feature can overcome this limi- 
tation .^JtectjU^^ the image plane and 
the workpiece plane onto each other. As can be seen from Fig- 
ure 4, the image becomes like a map that is seen from above 
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without perspective distortions. All images 55a - 55d can now 
be stitched together to create one huge meta-image 56. 




The repeatability and reproducibility of the photogrammetry 
5 approach has been analyzed by taking measurements of the same 
blank 60a, 60b under changed measurement conditions (Fig. 5) . 
Between two measurements the camera has been closed, the 
workpiece repositioned and the rulers and targets redistrib- 
uted to different locations. It can be seen from Figure 5 
10 that the resulting maximum deviation between two contours is 
well below 5 mm and generally much less. 

All the blanks used in this test have been measured on a co- 
ordinate measuring machine_J^^^ 

15 l ute accuracy o f the p hotogrammetry approach. The CMM meas- 
urements have been made using a 4 mm diameter cylinder as a 
touch probe. This implies that details finer than 4 mm cannot 
be reproduced correctly. Figure 6 shows the approximation of 
a rounded inside groove 71 by straight-line segments with one 

20 point in the apex. Since the groove has a width of only 9 mm 
the CMM cannot move the cylinder probe along the contour. 



Another source of deviation between CMM and photogrammetry 
may be insufficient sampling density on the CMM. The CMM op- 

25 erator may e.g. leave out points like fine "noses" at the 
corners of the blank because he regards them as irrelevant 
for form die construction. As these structures are modeled in 
the photogrammetric measurement, the photo points lie outside 
CMM reference. In the measurement statistics these points are 

30 out of tolerance but this does not really reflect the accu- 
racy of the method. 



Two independent photogrammetric measurements have been com- 
pared to the CMM reference measurement of a blank 80 
(Fig. 7). The three contour lines have been mapped (rotated 
and translated) onto each other so that the resulting devia- 
tion appeared to be as small as possible. Although this 
method may not produce the optimum result it can be seen from 
Figure 7 that the maximum error is about 5 mm for a part that 
is much larger than 1000 mm. The average error is obviously 
much smaller. 



A quantitative evaluation of the deviations between CMM and 
photogrammetry measurements is given in the following table: 



maximum deviation +3 . 17 mm 

minimum deviation -2.55 mm 

| max - min | 5.72 mm 

standard deviation a 1.16 mm 

points total 152 

points in tolerance 12 0 

points out of tolerance 32 



It has been verified that photogrammetry is a feasible ap- 
proach to replace CMMs in the measurement of sheet metal 
parts. There are several sources of error to be considered. 
As described above some artificial outliers are caused by the 
different sampling strategies on CMM and photogrammetry. One 
systematic error is that targets on the blank and targets on 
the surrounding area are assumed to lie in the same plane 
z = 0. This is not the case because of the blank material 
thickness. Typical blanks are 0 . 5 m to 2 m in length and less 
than 5 mm, preferably less than 2 mm thick. In order to re- 
move the mentioned error, targets of the surrounding area may 
be positioned on distance or equalizing layers that compen- 




sate for the thickness differences. Finally, the zoom lens of 
the camera is not optimal for the application. It cannot be 
guaranteed that the principal point remains constant during a 
set of pictures due to mechanical instabilities. 

5 

The present invention proposes the use of photogrammetric 
techniques for fast contour digitization. The goal was to 
speed up the determination of blank contours after manual op- 
timization during the tool tryout process. This task is 

10 two-dimensional in nature. A largely automated data process- 
ing allows the system to be used without specific knowledge 
about the underlying technology. In this context blank dig- 
itization can be looked at as a typical example of a measure- 
ment task with restrictions on floor space, part transport' 

15 and measurement time. 

The proposed system will cost less than 30,000. US$ including 
hardware and software. Considering the potential savings and 
the flexibility for further applications this can be called 

20 low cost. The attractive price is stemming from the use of a 
medium priced digital camera which will limit the achievable 
accuracy in the range of a few millimeters. Particular prepa- 
ration of the blanks such as cleaning, matting or painting is 
not required- A simple black background will provide enough 

25 contrast for secure detection of the blank contour. 

While the invention has been particularly shown and described 
with reference to the preferred embodiment thereof, it is 
well understood by those skilled in the art that various 
30 changes and modifications can be made in the invention with- 
out departing from the spirit and the scope of the invention. 



